Machine Life-Long Learning with csMTL Networks

Multiple task learning (MTL) neural networks are one of the better documented methods of inductive transfer of task knowledge (Caruana 1997). An MTL network is a feedforward multi-layer network with an output node for each task being learned. The standard back-propagation of error learning algorithm is used to train all tasks in parallel. The sharing of internal representation in the hidden nodes is the method by which inductive bias occurs between related tasks within an MTL network (Baxter 1996). Previously, (Silver & Mercer 2002; Silver & Poirier 2004) have investigated the use of MTL networks as a basis for developing machine lifelong learning (ML3) systems and have found them to have several limitations caused by the multiple outputs. In response to these problems, this article introduces context-sensitive MTL, or csMTL, and describes a ML3 system that uses these networks to perform long-term consolidation of task knowledge and the inductive transfer of prior knowledge during new learning.